When IT Teams Fail at the Basics: Why Big Software Projects Still Collapse
Software is everywhere — powering salaries, driving public services, enabling commerce — yet tens of billions of dollars in failures each year show we still can’t seem to build dependable systems at scale. A new article in IEEE Spectrum, titled “How IT Managers Fail Software Projects”, delivers a sobering verdict: we have failed to learn from decades of disastrous IT projects — and the costs keep rising. (IEEE Spectrum)
🚨 The Unsettling Reality
Since 2005, global IT spending has more than tripled — from roughly US $1.7 trillion to US $5.6 trillion in 2025 dollars. (IEEE Spectrum) Yet even with far more money pouring into IT, success rates for large-scale software projects remain disappointingly low. That’s not just frustrating — it’s dangerous. As software embeds into every facet of society, these failures impose heavy costs on individuals, institutions, and governments. (IEEE Spectrum)
The article argues that the root problem isn’t technical. It’s human. Management, process, unrealistic ambition, organizational inertia — these are the recurring themes in project failures. (IEEE Spectrum)
Two Cautionary Tales
Phoenix payroll system (Canada, 2016 — ongoing)
A government initiative to unify federal payroll and HR data across dozens of departments. Project leads cut corners: slashed budget, minimized integration testing, deferred important checks, and skipped proper piloting. (IEEE Spectrum) As a result, since its launch, hundreds of thousands of Canadian civil-servants have suffered paycheck errors, delays, and backlogs. Over CA $5.1 billion was wasted, and by early 2025 nearly 349,000 issues were still unresolved — many after more than a year. (IEEE Spectrum)
Horizon EPOS system (UK Post Office, 1999 — ongoing)
Built by Fujitsu and rolled out starting 1999, Horizon’s software was deeply flawed. Bugs were hidden, errors suppressed — and 3,500 post-office managers were wrongly accused of fraud, theft, and false accounting. (IEEE Spectrum) By 2015, nearly 900 had been convicted; 236 jailed. It took more than a decade of lawsuits, a public inquiry, and a media spotlight to begin unraveling the truth. Even now, after massive societal damage, the Post Office still uses the system while planning a £410 million replacement. (IEEE Spectrum)
Why It Keeps Happening
- Recycled mistakes, ignored lessons. Many of the underlying causes — scope creep, poor testing, lack of oversight — are well known and documented. Yet IT managers often claim “our case is different,” brushing aside the lessons. (IEEE Spectrum)
- Complexity overload. Modern software projects often involve deep integration, multiple interfaces, and a tangled web of dependencies. If project goals and risks aren’t clearly defined — or constantly shift — the outcome is predictable: chaos. (IEEE Spectrum)
- Organizational and human failures. Often it’s not code that breaks — it’s processes, culture, and accountability. Unrealistic deadlines, poor governance, and ineffective communication often doom projects before they begin. (IEEE Spectrum)
- Legacy inertia. Even when organizations know their systems are broken, replacing them feels too risky or costly — particularly after seeing projects like Phoenix or Horizon implode. So outdated, brittle systems stay live — with predictable failures. (IEEE Spectrum)
Attempts to remedy this — through modern practices like Agile and DevOps — have shown some success. But these alone aren’t silver bullets. Without discipline, leadership commitment, and structural reform, even agile-based projects can fail. (IEEE Spectrum)
Why This Matters — Especially in an AI-Saturated World
As we increasingly rely on software — and AI-powered systems — in domains like welfare, payroll, health, public services, transport, finance, the stakes are higher than ever. A buggy payroll system isn’t just an inconvenience — it’s financial instability for tens of thousands of families, mental anguish, lost trust. As the article warns: when software fails at scale, society pays. (IEEE Spectrum)
We can no longer afford to treat software failures as “just another bug.” We need structural accountability, mature governance, and a commitment to learning from past mistakes.
Glossary
- IT project mismanagement — Failures caused not by code bugs, but by poor planning, unrealistic scope, lack of testing or oversight, and weak organizational governance.
- Scope creep — When the requirements or goals of a project continuously expand beyond its original boundaries, usually without corresponding increases in resources or schedule.
- Legacy system — Older software systems that continue to be used, often with outdated architecture or dependencies, which are difficult to maintain, update, or replace.
- Agile / DevOps — Software-development methodologies aiming to deliver projects iteratively with frequent testing, feedback loops, and rapid deployment — intended to reduce risk compared to traditional waterfall approaches.
Takeaway
More money, better tools, smarter engineers — yet we still trip over the basics. The recurring disasters of Phoenix, Horizon, and many other IT meltdowns prove that software reliability is less about code than about discipline, governance, and humility. As software becomes more pervasive — and critical — we must stop treating failure as inevitable and start treating it as preventable.
